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Of all the issues discussed at Alife VII: Looking Forward, Looking Backward, the issue 
of whether it was possible to create an artificial life system that exhibits open-ended 
evolution of novelty is by far the biggest. Of the 14 open problems settled on as a result 
of debate at the conference, some 6 are directly, or indirectly related to this issue. 

Most people equate open-ended evolution with complexity growth, although a priori 
these seem to be different things. In this paper I report on experiments to measure the 
complexity of Tierran organisms, and show the results for a size-neutral run of Tierra. 
In this run, no increase in organismal complexity was observed, although organism size 
did increase through the run. This result is discussed, offering some signposts on path 
to solving the issue of open ended evolution. 



1. Introduction 

An open discussion forum was conducted recently at Alife VII to draw up a list 
of open problems that might guide the field of artificial life in the coming years, 
styled on the famous Hubert problems that guided much of Mathematics in the 20th 
century. The resulting lisiu contains 14 open problems, divided into 3 categories: 
How does life arise from the non-living; What are the potentials and limits of living 
systems; How is life related to mind, machine and culture. 

The issue of open-ended evolution can be summed up by asking under what 
conditions will an evolutionary system continue to produce novel forms. Artificial 
Life systems such as Tierra and Avida produced a rich diversity of organisms ini- 
tially, yet ultimately peter outjj By contrast, the Earth's biosphere appears to have 
continuously generated new and varied forms throughout the 4 x 10 9 years of the 
history of life. There is also a clear trend from simple organisms at the time of the 
first replicators towards immensely complicated organisms such as mammals and 
birds found on the Earth today. This raises the obvious question of what is missing 
in artificial life systems? 

This issue is touched on most directly in problem 6 — "What is inevitable in 
open-ended evolution of life", but in fact is intimately connected with problems 1, 

2, 3, 5 and 7 also. 

The issue of complexity growth is related tQihat of open-endedness. Complexity 
is related to information in a direct manner ,E3o in a way to be made more precise 
later in this paper. Loosely speaking, available complexity is proportional to the 
dimension of phenotype space, and an evolutionary process that remained at low 
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levels of complexity will quickly exhaust the possibilities for novel forms. However, 
intuitively, one would expect the number of novel forms to increase exponentially 
with available complexity, and so perhaps increasing complexity might cease to 
be important factor in open-ended evolution beyond a certain point. Of course, 
it is by far from proven that the number of possible forms increases as rapidly 
with complexity as that, so it may still be that complexity growth is essential for 
continual novelty. 

For the purposes of this paper, we will examine the possibilities for open-ended 
growth in complexity in artificial life, since this most closely resembles the evolution 
of our biosphere. Tt i s worth bearing in mind that the biosphere is subject to the 
anthropic principled while no artificial life model is. Whether this fact is important 
in the generation of complexity is a question worthy of artificial life studies. 

Results will also be reported of an experiment to measure the complexity of 
Tierran organisms. Whilst it is widely believed that the version of Tierra used here 
does not exhibit open ended complexity growth, the Tierran team are hoping to 
produce a version of network Tierra that does. The importance of this work is to 
demonstrate the feasibility of these sorts of experiments on simple ALife systems, 
before attempting more complex ones. 



2. Complexity 

The concept of complexity as a measurable quantity has been subject to a lot 
of debate. This is particularly due to the fact that there are numerous ways of 
ordering things based on physical quantities that agree with our subjective notion of 
complexity, within particular application environments. Examples include organism 
size, numbers of parts and number of different cell types in the arena of complexity 
of animals, to name just a few.@ A good discussion of these different types can be 
found in Edmonds .! Unfortunately, these measures typically do not extend to other 
application areas, where most of the complexity may be encoded in the connections 
between the parts, or in the levels of hierarchy. 

There is, however, a unifying concept that covers all notions of measurable com- 
plexity, and that is information. Information was first quantified in the landmark 
work of Shannonll3 in the late 1940s, however in this case the semantic content 
of information was ignored, since the purpose was to measure the capacities of 
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communication channels. The next theoretical step occurred in the 196C 
advent of algorithmic complexity, also known as Kolmogorov complexity) 
gorithmic complexity, the complexity of a description is defined to be the length of 
the shortest program that can generate the description, when run on a particular 
universal Turing machine (UTM). This complexity is closely related to the Shan- 
non entropy in the following way. Consider the set of all infinite length descriptions 
expressed as binary strings. A halting program of length £ will be equivalent to 
all infinite length descriptions having the same prefix of length £, and so will have 
probability 2~ £ of being picked at random from the set of all descriptions. This 
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gives rise to the so called universal prior measure, defined a E 

Pu(x)= J2 2 ~ e(P) > f 1 ) 

p:U computes x from p and halts 

where U is the reference universal Turing machine, and £(p) is the length of program 
p. The Shannon entropy computed using this probability distribution differs from 
the algorithmic complexity by at mosLa constant that depends only on the Turing 
machines chosen, not the description]!!] 

C{x) = - log 2 P v [x) = K v (x) + C v , 3C V > (2) 

where Ku(x) is the algorithmic complexity of x. 

There are two key problems with algorithmic complexity. The first is the arbi- 
trariness of the reference Turing machine U. The second relates to the behaviour 
of random strings — the algorithmic complexity of a random string is at least as 
great as the string length itself, ie has maximal complexity. This is in conflict with 
our common sense notion of random strings having no information at allJZ 

The first problem is really a non-problem. All information is context dependent, 
with the context being defined by the receiver or interpreter of a message. See 
Adamitlfor an explicit discussion of this issue. In the case of algorithmic complexity, 
the interpreter is the reference UTM, which must be agreed upon in the context of 
the discussion. 

The second issue can be dealLwith by broadening the concept of algorithmic 
complexity in the following way:E2l given an interpreter of information that defines 
the context, the interpreter must classify all possible messages into a countable, 
possibly finite set of valid meanings. The UTM classifies messages into equivalent 
programs. A human interpreter will have a somewhat different scheme of classifying 
things. With respect to the particular issue of random strings, a human being will 
classify all random strings as identical — all gibberish. The obvious generalisation 
of complexity is: 

C{x) = lim s log 2 N — log 2 u>(s, x) (3) 

s — >oc 

where C{x) is the complexity (measured in bits), N the size of the alphabet used to 
encode the description and lu(s, x) the size of the class of all descriptions equivalent 
to x and of length less than s. This definition allows for infinite strings to be 
measured, provided the above limit exists. Random strings are dense in the set 
of all possible strings, so oj(s,x) — > N s as s — ► oo. Random strings have zero 
complexity. 

To see that this definition of C(x) converges for finite descriptions (ie ones for 
which the interpreter "halts" and gives an interpretation after a fixed number n 
characters), note that all strings having the same n character prefix are equiva- 
lent. Therefore, ui(s, x) > N s ~ n , so C(x) < n\og 2 N. Equality only holds iff x is 
incompressible. 
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In the case where the interpreter is the UTM U, each finite prefix bitstring p 
equivalent to x contributes 2 s ~ e ( p ' to u)(s,x). So 

to(s,x) = r ~ e(P) = 2Sp u(%) 

p:U computes x from p and halts 

=> C{x) = - Van Pu{x) 

s — >oc 

so C(x) reduces to the usual complexity measure in equation (Q). 



3. Complexity in Tierra 

Now that complexity has been defined in a general sense, we need to work out an 
appropriate context for discussing the complexity of Tierran organisms. Tierra is 
an artificial life system created by Tom Raylij where the artificial organisms are 
computer programs written in a special assembler-like language. The equivalence 
principle for defining the context is phenotypic equivalence, ie when two Tierran or- 
ganisms behave identically, even if they contain a different sequence of instructions. 
Adami's group pioneered this technique with ^OTdaJira a similar artificial life system 
inspired by Tierra. In Avida, the situation is particularly simple, as organisms have 
only one phenotypic characteristic, namely their reproduction rate. To make mat- 
ters more interesting, the Avida group set the creatures a range of computational 
tasks to attempt — success at these tasks is rewarded by extra CPU resources. The 
success or otherwise of the creatures at these tasks further distinguishes between 
Avida phenotypes. 

Tierra presents considerably greater difficulties than Avida in that Tierran or- 
ganisms can interact with each other via template matches or simply "falling off 
the end of the code" . So, the Tierra phenotype can be categorised by examin- 
ing the interactions between all possible phenotypes. Since genotype space is so 
huge, what better way of searching for viable phenotypes than using a genetic al- 
gorithm, namely taking the results from a l ong Tierra. run. These are then pitted 
against each other in pairwise tournamentscinij in a specially crafted simulator 
called miniTierraEQ that can perform the tournament nearly 1000 times faster than 
Tierra itself. Once all the tournaments have been completed, the organisms are 
sorted into phcno typically equivalent classes, resulting in a small list of 26 archety- 
pal organisms. These are labeled by the genotype with the earliest creation date. 
Table 1 lists the archetypes, along with their creation time, non-volatile site count 
(to be explained later) and measured entropy. 

Having now established an archetype list, we need to estimate the density of 
neutrally equivalent genotypes in genotype space in order to apply the complexity 
formula ([|) . Unfortunately, simply sampling the whole of genotype space is compu- 
tation infeasible, as the overall density is low — an organism having complexity of 
60 instructions (300 bits) will have a density of 32 -60 requiring many more samples 
than there are nanoseconds in the history of the universe^. We can, however, ex- 

a Tierra uses 5 bit instructions, so there are 32 possible instructions at each site of the digital 
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plore the neutral network of phenotypic equivalents connected by single mutations. 
Unfortunately, this also is computationally infeasible — take our previous example 
of an organism of length 80 instructions, having a complexity of 60 instructions. 
The neutrally equivalent set will have size 32 20 genotypes, each of which needs to 
be checked. 

What we can do is a hybrid method: begin by exploring the neutral network 
out to a certain cutoff value (usually 2 hops in this experiment, however several 
of the cheaper organisms were computed to higher cutoff value to check on con- 
vergence). Where the network is thickly branching, perform a Monte-Carlo style 
sampling of that subspace, since this is likely to correspond to part of the genome 
that contains little information. Where the network has only a few branches, then 
continue following the network branches. The threshold between the two strategies 
is determined by some heuristic estimates of how expensive each approach is likely 
to be. Basically the density of neutral variants in a genotype subspace is assumed to 
be proportional to the number of immediate neutral neighbours in that subspace. 
This then entails both the cost of traversing the network (which rises exponentially 
with neutral density) and the cost of performing Monte Carlo sampling, which falls 
with increasing neutral density. 

The code and data used for this experiment is available from 
^ttp://parallcl.hpc.unsw.cdu.au/rks/software/cco-ticiTa-l.l.tar.gz . The use of the 



software is briefly documented in the README file. 

We end up with a probabilistic estimate of the density of phenotypic equivalents, 
and hence the complexity by (|^). This estimate should converge as more of the 
neutral network is searched. 

Since the first thing to compute is the number of nearest neutral neighbours at 
each site, a simple complexity estimate can be found: classify sites as hot if every 
possible mutation leaves the organism neutrally equivalent, and cold if any mutation 
changes the phenotype. The number of cold sites could be called the non-volatile 
site count, denoted Cj^y. A slightly more sophisticated calculation is to compute 

Css=^-X^ lo g32"i (4) 

i=l 

where 71$ is the number of mutations at site i on the genome that lead to differing 
phenotypes, which we can call the single site entropy. For a completely cold site, 
rii = 1, and for a hot site n,; = 32. Formula (||) takes into account shades of grey 
in "site temperature" . Adami et al. use this measurement in their work.EI Equation 
(^) is clearly an underestimate of the amount of complexity defined in (|^) , as is the 
non- volatile site count. 



4. Results 

Table 1 shows the creation time (in millions of instructions) , computed single site 
entropies and complexity measures estimated by truncating the neutral network at 



genome 
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two hops (denoted C2) for each of the phenotypes found in this sample run. Figure 
|l] shows the same results in graphical form, with the creation time plotted along 
the horizontal axis, and the organism length, non-volatile site count Cj^y and C2 
plotted as separate points. There are several key points to note: 

• Over this Tierra run of 6 x 10 9 instructions, there is no sign of complexity 
increase, although lengths of the organisms do increase from 80 initially to 
526 instructions long by 1.4 x 10 9 instructions executed. CPU resources are 
divided amongst the organisms in a size neutral fashion (ie proportional 
to organism size) to allow this growth in genome size to happen. This is 
not an especially lengthy Tierra run, however, and perhaps real complexity 
takes much longer accumulate in the junk parts of the genome. 

• The non-volatile site count Cj^y is a good proxy measure of the total 
complexity. The biggest discrepancy in this database occurred for 0194aag, 
and the real complexity was only 58% higher. This is good news, as it is 
far more tractable to compute Cj^y, than the full complexity. The full C2 
computation for this dataset of organisms required nearly 8 CPU years of 
contemporary processor time, as opposed to about 3 CPU hours for Cj^y- 
Fortunately, it is a highly parallel problem, and was computed within about 
3 months on a major high performance computing facility. 

• The precise definition of the phenotypz has a big iiifliience over complexity 
values. As has been argued by Adamitl and myself,E2l this is highly context 
dependent. No particular definition is wrong per se, however some defi- 
nitions will be better than others. I hope, dear reader, you consider my 
definition based on pairwise ecological interactions to be a good definition, 
and that therefore the complexity measures obtained are useful. It is known 
that Tierran organisms can exhibit what is called social hyperparasitismx3 
This is necessarily a 3- way interaction between Tierran organisms — hyper- 
parasitism refers to stealing another organism's CPU to reproduce itself, 
and social refers to the fact that multiple organisms must cooperate to 
perform this feat. However both of these properties show up as distinct 
patterns amongst 2 way interactions - the hyperparasite still manages to 
interfere with its prey's reproductive capability, even if it is unable to gain 
from the act alone, and mutual interaction with members of its own kind 
show up also. 

5. Discussion 

Thus far, the focus of this work has been on presenting a practical method of com- 
puting the complexity of Tierran organisms. However, it has been long recognised 
that neither Tierra nor Avida exhibit open-ended evolution evolution. It is thought 
that these systems rapidly adapt to and exhaust the possibilities of a fairly simple 
environment. Adami et al. have approached the problem by artificially increasing 
the complexity of the environment by specifying a sequence of arithmetical oper- 
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Fig. 1. Plot of the results for the archetypes. The horizontal axis plots the time of first appear- 
ance, and for each organism is plotted the length, non-volatile site count Cj^y and the complexity 
C 2 . 



ations that the digital organisms can attempt. Success is rewarded by extra CPU 
time. In these circumstances, they have shown increasing complexity of the or- 
ganisms over timelrcl Ray, on the other hand, is attempting to exploit naturally 
occurring environmental complexity provided by the Internet, Ell and hopes to en- 
gineer a "Cambrian explosion" . To date, his team has had mixed success — whilst 
they have managed to get organisms to persist in a multicellular state under muta- 
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tional load, as opposed to_ceverting to a single celled, they haven't achieved their 
aim of complexity growth Jl3 

The obvious next step in research is to attempt to measure the complexities of 
network Tierra organisms. It is by no means clear that this is tractable. The success 
of this approach with the original Tierran and Avidan organisms may well be due 
to the simplicity pi the environment. 

As I observedjHa the natural biosphere operates under the constraints of the an- 
thropic principle. This means that we must necessarily observe a path to increased 
complexity in the biosphere. Furthermore, there are some statistical regularities 
in evolutionary history that indicates the evolution of the biosphere to be an ex- 
tremely rare process rather than an inevitable one.S The universe appears to be 
performing a quantum search algorithm to find the precise conditions required to 
generate intelligent life. Probably the way forward is to perform a similar sort of 
"scattergun" approach, using the power of quantum computers if and when they 
become available. 

A key step in doing this is to generate a process that adaptively recognises 
complexity, since it will be impossible to include humans in the loop, even when 
run on conventional computing platforms. To this end, techniques developed for 
data mining^ should prove useful. As a curious twist, artificial life techniques are 
being successfully applied to the domain of data miningE! Developing a coevolving 
system observer is something that can be started now with present technology. 
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